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ABSTRACT 

This report is an attempt to conceptualize a 
comprehensive frameworJc for evaluating vocational education quality. 
It is intended to inform policymaJters of the breadth of the elements 
of quality and to suggest an integrated framework for assessing 
quality to vocational education evaluation theoreticians. Chapter 1 
offers preliminary considerations. Chapter 2 introduces and defines 
the elements of quality related to educational outcomes that were 
identified through the model-building process described in Chapter l. 
Two general types of outcomes are included: individual and societal, 
with a subset of societal outcomes specific to employers. Chapter 3 
parallels chapter 2 in structure, but focuses on the elements of 
quality related to the educational experience. The educational 
experience is divided into two domains: school context and school 
process. Chapter 4 presents some issues and guidelines regarding the 
development of quality indicators for the elements within the 
proposed model. General evaluation approaches and specific models are 
discussed as they relate to developing indicators for the elements of 
quality in vocational education. Thirty-two references are appended. 
(YLB) 
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FOREWORD 



Traditionally, vocational education program quality has been defined (1) in terms of tho output 
of the program (e.g , training-related jobs obtained by former students, satisfied employers) or 
(2) as the process of the program (e.g., time spent on learning tasks, skill content addressed) Both 
of these are logically related to quality, but neither one by itself provides an explanation of quality 
that is sufficient to influence policy toward quality improvement. 

This publication provides a discussion of the elements of quality in vocational education as 
viewed in the context of formal education. Reviews of these elements by knowledgeable practi- 
tioners and recommendations related to the development of quality indicators are also included It 
is an attempt to conceptualize a comprehensive framework for evaluating vocational education 
quality. The report is intended to inform policymakers of the breadth of the elements of quality and 
to suggest an integrated framework for assessing quality to vocational education evaluation 
theoreticians. 

This project was conducted in the Evaluation and Policy Division of the National Center for 
Research in Vocational Education under the direction of N L. McCaslin, Associate Director. Many 
people made significant contributions in the course of its completion. We wish to thank for their 
participation in our field discussions Dr. Nellie Dry, Dr. George Mehallis, Charlotte Hayes, and 
David Hall, associated with Broward Community College in Ft. Lauderdale, Florida: John Ryan, 
Ron Smith. Joe Fabing, and Billy Keith Butt, associated with Tn-County Joint Vocational School in 
Nelsonville, Ohio; Dr. Dwight Davis. Marvin Bausman, Dan Sczygelski, Larry Post, and Cindy 
Pogachnick. associated with North Central Technical Institute in Wausau, Wisconsin; Dr. Jim 
Frasjer, Nelson Whitemen, Harold Darling. Harold Pearson, and Beth Allen, associated with Patter- 
son Cooperative High School in Dayton, Ohio; and Ray Bertleson, John Miller, Marilyn Gordon, 
and Matt Meulock, associated with Westerville South High Scnool in Westerville, Ohio. 

We also acknowledge the contributions of National Center staff members Roy Butler, Ida 
Halasi, N. L. McCaslin. Allen Wiant, and Jane Reisman and the field work of Cathy Warmbrod and 
Jim Hamilton, which was instrumental in the development of our model. 

David L. Passmore. professor of vocational education. The Pennsylvania State University, 
George Quarles, former Director of Vocational Education for New York City, and Robert H. 
McCabe, president, Miami-Dade Community College, provided helpful criticisms and suggestions 
related to the overall context and outcomes of our conceptual framework 

We wish also to thank the project staff— Paul Campbell, project director. Jim Hamilton, Cathy 
Warmbrod, and Phyllis Panzano for their work in preparing the publication. And finally, we thank 
Jeanne Desy for her comments related to the organization of the report, Mary Beth Dauner for her 
effort, cooperation, and patience in typing the manuscript with its many revisions, and Janet 
Kiplinger, under whose supervision the publication was edited 



Robert E. Taylor 
Executive Director 
National Center for Research 
in Vocational Education 
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EXECUTIVE SUMMARY 



The elements of quality proposed in this publication have been deri' ed from a diverse body of 
literature and experience. We have attempted to incorporate various findings related to attributes 
of quality education and programs into a general yet comprehensive conceptual framework in 
order to broaden our understanding of the elements of quality vocational education. It will con- 
tribute to an increased awareness of the major components influencing the quality of education. 
This publication is intended to inform policymakers of the breadth of the elements of quality and 
to suggest an integrated framework for assessing quality to policymakers and vocational educa- 
tion evaluators. 

For purposes of this effort vocational education was defined as school-related activities that 
occur in secondary schools and 1- and 2-year programs in postsecondary schools; vocational 
education was considered to be a subset of general, formal education, which in turn is a subset of 
education coming from a variety of life experiences. 

Our framework for the elements of quality grew out of a multiphased process that included 
reviews of related literature, expert input, and key informant interviews. The literature provided 
preliminary structure to the framework that was further refined and defined by expert opinion and 
key informant commentaries. 

Some vocational education has been directed by law to be evaluated in terms that include job 
placement in training-related employment, employer satisfaction, and quality and availability of 
Instructional offerings. This effort, however, initially explored a variety of desirable outcomes that 
may help to define high quality vocational education programs and subsequently identified the 
components of the educational environment that may support attainment of those outcomes. The 
outcomes were therefore considered as goals and the structures and processes within the educa- 
tional environment were perceived as means for achieving the goals. The resulting framework for 
the elements of vocational education quality (figure 1) was conceptualized as a complex environ- 
ment converging on the educational experience, driven by desired individual and societal out- 
comes, and influenced by environmental inputs such as community setting and labor market 



Elements of Quality Outcome 

Two general types of outcomes are Included in our framework individual and societal with a 
subset of societal outcomes specific to employers. Four elements of quality relating to individual 
outcomes were identified and defined Competency development encompasses the development 



conditions. 



The Framework: Elements oi Quality 
Vocational Education 
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of work-related skills (both occupational and job specific), basic skills such as verbal and mathe- 
matical, and general problem-solving ability. Positive interpersonal interaction involves the ability 
of people to have positive and mutually supportive relations with others in the home, the work- 
place, and the community. A sense of efficacy is an attitude of confidence in one's ability to utilize 
skills to achieve goals. The ability to manage a large array of tasks without having to depend upon 
someone else is designated as a sense of independence These are the four general elements of 
quality individual outcomes included within our conceptual framework 

Social stability and employer needs are the primary elements of quality withm the societal 
outcome portion of our framework Social stability does not imply social stagnation, rather, it 
means gradual, orderly change Provision of essential services, quality of vi/ork products, resources 
for self-support, and societal improvement are the subelements that support social stability 

We also examined a subset of societal outcomes specific to employers To be worth the 
investment from the employer's point of view, vocational education should contribute to available 
skills, reduced training time, and high productivity These are the three elements of quality related 
to employers needs Our definition of quality vocational education outcomes includes, but is not 
limited to, employment-related skills. Rather, vocational education of a positive quality must also 
enhance the development of desirable or even essential skills or dispositions that are not directly 
employment related 

Eltmtnts of Quality within 
tht Educational Experience 

Taking a systems perspective, the processes and structures that define the elements of quality 
within the vocational education experience should relate to intended individual and societal out- 
comes. Our mo< el divides the educational experience into two domains, school context and 
school process. School context deals with both physical and organizational aspects of the school 
unit Personnel, facilities, equipment, policies, and formal organizational structure all fall within 
school context. 1 he elements of quality within school context that are formally included m our 
model are school setting, classroom setting, teacher skills, teacher characteristic^ and peer group 
characteristics. Figuratively speaking, school context provides the seating for sch. process, 
which provides the "action " Estimations of quality of school process would consider such things 
as course content; course-specific equipment; interaction between teacher, student, and materials, 
and level of formal student involvement in the vocational education program The elements of qual- 
ity formally included within school process are teacher, student, and material interaction, instruc- 
tional content and material: and student utilization of instructional process 

These elements of quality within the educational experience should relate to intermediate 
objectives that, m turr,. -hould support attainment of desired outcomes, the linkage should be sys- 
tematic between quality of experience and quality of outcome in order to maintain and replicate 
effective programs and make modifications as necessary. 



Assessing the Quality of Vocational Education 



The elements of quality within our framework are generalizable across school unit and pro- 
gram However, the specific indicators of quality associated with them may vary across school and 
program and are likely to vary over time 



Considerable effort has been expended in developing evaluative strategies to facilitate the 
assessment of quality. Evaluation models should combine- structural, process, and outcome 
approaches in ordcfto assess the range of elements of quality withtn our framework 

Specific indicators of quality need to be developed at the individual school unit level It is not 
practically possible to tabulate exhaustively all the indicators related to the elements of our model, 
nor is it financially practical to support an evaluation system capable of gathering and analyzing 
such da* in a complete form A process for identifying and prioritizing subelements within the 
model is suggested as a practical starting point for developing indicators A combination of quan- 
titative and qualitative methods is suited to a comprehensive evaluation of quality, and both 
normative and empirical approaches should be drawn upon for developing indicators 

In summary, this discussion presents a justification and set of procedures that will permit an 
Integrated consideration of the elements of vocational education quality We have advocated con- 
sidering a broad set of elements of quality rather than a narrowly defined or segregated set. so that 
the evaluation of vocational education reflects its contribution to a set of living experiences in 
addition to employment 
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CHAPTER 1 
INTRODUCTION 
Somt Prtlimlnary Comlctoratlons 



WMt Is This PuMteatton About? 

When determining whether or not a pro- 
gram of instruction is producing the desired 
outcomes, it is first necessary to decide what 
those outcomes should be. Vocational educa- 
tion has been directed by law to be evaluated 
in terms of job placement in training-related 
employment and employer satisfaction. 
Atthough both of these outcome? Are reason- 
able as well as legally required, there is a 
large body of opinion that extends the 
desired outcomes beyond that limited set. 
This publication advocates such an expan- 
sion for the purpose of evaluation, explores a 
variety of potentially desirable outcomes that 
may help to define a high-quality vocctiona! 
education program and identifies those 
components of the educational experience 
that may support attainment of those 
outcomes. 

The Clements of vocational education 
quality may be found in two places^the out- 
comes of the educational experience and the 
educational experience itself. In this discus- 
sion we present a tentative set of outcome 
goals* with some description of their nature 
and justification for their selection. The justi- 
fication relies upon normative premises, 
empirical and theoretical evidence, and, in 
some cases, intuitive logic. Because the pro- 
cesses and structures that presumably define 
the educational experience should relate to 
the intended outcomes, the outcomes are 
considered first. A similar consideration of 
the processes will be the topic of a subse- 
quent chapter. 

are attempting to conceptualize a 
genera/ yet comprBhensivB framework for 
assessing vocational education quality. It will 
contribute to an increased awareness of the 



breadth of the components that influence the 
quality of education. 



What Is Vocational Education? 

Before proceeding with a sear^;n tor ele- 
ments of quality in vocational education, 
some fences must be erected around the 
varied terrain of ideas and activities to which 
the \atHi\ 'Vocational education** is attached. 
From a narrow point of view, vocational edu- 
cation is considered by some as only those 
school-related activities that are the recip- 
ients of Federal funds under the vocational 
education acts t)eginning with the Smith- 
Hughes Act in 1917. On the other hand, it 
may be considered as any school-related 
activity, through graduate and professional 
schools, that provides skills for employment 
in a specific field— from machining to medi- 
cine, or from cooking to counseling. For pur- 
poses of this inquiry, we have elected to limit 
consideration to those school-related activi- 
ties that take place in secondary schools and 
in 1- or 2-year programs in postsecondary 
schools. These school activities that we clas- 
sify as vocational education include— but as 
we intend to show subsequently— are not 
limited to programs that produce salable 
skills specific to general classes of occupa- 
tion. They may or may not be supported by 
Federal funding. 

Where Does Vocational Education Fit? 

We consider vocational education to be a 
subset of general, formal education, which, in 
turn, is a subset of education coming from a 
wiOe variety of life experiences. At the 
secondary level, it is part of a universal and 
obligatory educational experience. It is uni- 
versal in most Western democracies because 
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provision is made in the laws n/ these States 
to provide it for nearly everyone. The most 
usual exception occurs where some 
extremely severe mental or physical problem 
renders formal education impractical. Educa- 
tion is an obligation because most of these 
States require some level of compulsory 
attendance. 

At the postsecondary level, the obliga- 
tory feature of education is completely 
relaxed. The univeisal quality is also much 
less apparent, although the general availabil- 
ity of public funds to support postsecondary 
institutions, at least in the United States, 
reflects a commitment to broaden the sccess 
of individuals to continued education. 



Wlial thouM Be the Outoome of General, 
Fonnal Education? 

To understand where vocational educa- 
tion fits within this scope of general, formal 
education, \* is necessary to examine what 
the outcomes of general, formal education 
are expected to be. Frani(ena (1974) de- 
scribes these outcomes as the formation of 
what he calls ''dispositions" that include abili- 
ties, habits, beliefs, sicills. and bodies of 
knowledge. 

It \n beyond the scope of this discussion 
to elaborate upon the basic normative prem- 
ises underlying the selection of the disposi- 
tions that seem to t>e most universally valued 
in American society. Rather, those that have 
broad representation in the literature, in the 
media, and in practice suggest what appears 
to be the society's underlying values. If we 
consider both practice and commentaries, 
the currently existing values, as well as those 
desired, have a chance to emerge. 

There seems to be general agreement 
that speaicing. reading, and writing the major- 
ity language, as well as handling basic quan- 
titative problem solving, are fundamental 
elements of the dispositions. / jast some 
level of comprehension of the procedures of 
science and the political organization of the 



Nation. State, and community are also 
generally accepted. Some National reports 
(e.g., by the National Commission on Excel- 
lence. The Twentieth Century Fund) have 
additionally emphasized such other disposi- 
tions as computer literacy and foreign 
languages. 

That acquisition of the foregoing disposi- 
tions (e.g.. language, math) is necessary for a 
good and productive life seems to t>e sup- 
ported by the ot>$ervable. normative prem- 
ises. There is no general agreement about 
how all of these basic sicills should be pro- 
vided, but the presence of some elements of 
language, government, science, and math 
curriculum in virtually all high schools 
underscores the importance attached to for- 
mal study of these skills by American society. 



All societies are made up of individuals 
and of institutions that, at the time ot their 
founding, were intended to provide some 
form of service to individuals. {This purpose 
may have t)een lost to self-perpetuation as 
the institutions developed.) American 
society, in which the form of vocational edu- 
cation t>eing examined is set. has a long 
tradition of emphasis upon the individual. 
Illustrative are the writings of Thomas 
Jefferson on the need for an educated citi- 
zenry. Abraham Lincoln's eloquent reference 
to ''government for the people, by the people, 
and of the people" in the Gettysburg address, 
and John Dewey's concern for the role of the 
individual, expressed in the following Quote- 
Such societies were found to be denr;o- 
cratic in quality, because of the greater 
freedom allowed the constituent 
members, and the conscious need of 
securing in individuals a consciously 
socialized interest, instead of trusting 
mainly to the force of customs operating 
under the control of a superior class. The 
sort of education appropriate to the 
development of a democratic community 
was then explicitly taken as the criterion 
of the further, more detailed analysis of 
education. (Dewey I9l6. p 322) 
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In this society, education is focused upon 
individuals and seems to have two basic 
goals--to help individuals maximize their 
welhbeing in a broad sense and to provide a 
stability in the society through ensuring the 
well-being of its members collectively. 

An argument may also be made that 
worl( is necessary for the good and produc- 
tive life. As John Qardner (1961) puts it. "The 
best-icept secret in America today is that 
people would rather work hard for something 
they believe in than enjoy a pampered idle- 
ness.** Although the context of Gardner's 
remaric is the need to invest human energy in 
improving society in general, it applies to 
vocational education, which can and does 
address directly a specific inclination to woric 
that serves ooth societal and individual ends. 
One can easily recall numerous expressions 
of the desire to woric ranging from expecta- 
tions to continue wori(ing expressed by lot- 
tery winners, through the search for part-time 
jobs by retired persons, to the pride with 
which woricers display tangible results of 
their efforts. Vocational education provides 
specific training that enables people !o woric. 
and thus addresses the individual's need 
represented by these expressions as well as 
society*s need for the outcomes of an indt- 
viduars woric. 

Vocational education cannot be pursued, 
however, at the expense of some acceptable 
level of the broad basic dispositions that 
were detailed earlier. That these are generally 
valued is evidenced by their prevalence and 
also by their support by National sounding 
boards. To quote Dewey (1916) again 

Put in concrete terms, there is danger 
that vocational education will be inter- 
preted in theory and practice as trade 
education: as a means of securing tech- 
nical efficiency in specialized future 
pursuits. 

Education would then become an 
instrument ot perpetuating unchanged 
the existing industrial order of society, 
instead of operating as a means of its 



transformation. The desired transforma- 
tion is not difficult to define in a formal 
way. It signifies a society in which every 
person shall be occupied in something 
which maices the lives of others better 
worth living, (p. 316) 

The National Commission on Secondary 
Vocational Education seconds Dewey s 
concern^**Vocational education should not 
dilute the student's general education nor 
disenfranchise certain groups through selec- 
tive enrollment" (1984. p. 11). Therefore, 
vocational education as we have defined it is 
part of the broader concept of education with 
its general goal of individual and societal 
well-being. It shares with general, formal 
education the dispositions that are presumed 
to lead toward these two aspects of 
well-being. 

Conceptual Framework 

Before going further, it may be useful to 
consider the overall setting in which voca- 
tional education operates. Figure 1 has been 
prepared to illustrate one schematic of the 
conceptual frameworii. Briefly, figure 1 dis- 
plays three critical components of the overall 
setting. The left-hand panel displays the 
major dimensions of the context. These are 
conditions that exist independently of. and in 
most cases, prior to. participation in voca- 
tional education. Although the figure does 
not mal;e it explicit, the general context con- 
tinues to operate beyond formal education 
and influences independently the effective- 
ness of the outcomes in accomplishing the 
goals that underlie a conception of quality. 
The central panel displays the major dimen- 
sions of the vocational education setting. 
Three categories comprise this panel. They 
are the school-specific context, the teachmg- 
learning process, and the formal utilization of 
that process by the student. The right-hand 
panel displays a set of outcomes. These are 
divided into two major cetegories. Individual 
and societal. 

The outcomes panel is discussed first in 
this report. Discussion of the school setting. 
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CONTEXT 



EDUCATIONAL 
EXPERIENCE 



Family Background 



Personal Characteristics 



Community Setting — 



Labor Market Conditions 



School Context 



School Setting 
Classroom Setting 

Teacher Skills 
Teacher Characteristics 

Peer Group Characteristics 



I 



Student Utilization 
of lnstructk>nal Process 
(Patterns of Participation) 



Teacher, Student, and 
Material Interaction 



I 



Instructional Content 
and Equipment 



School Process 



Figure 1 . Framework identifying the elements of vocat 
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VOCATIONAL 
OUTCOMES 



Individual 

CompeterKy development 

Basic skills 

lnformatk>n skills 

Problem solving skills 

Employment skills 

Job-specific skills 

Career awareness 
Positive interpersonal interactions 
Sense of efficacy 
Sense of independence 

Enrichment 

Self-helping skills 



Societal 

Social stability 
Essential services 
Quality of work products 
Resources for self-support 
Societal improvement 

Employer needs 
Available skills 
Reduced training time 
High productivity 



education quality 



tMching-learning process, and student utili- 
zation will follow The overall approach can 
best be understood, however, if the concep- 
tual framework is kept in mind. 

The approach taken here assumes a 
functional role for education, rather than a 
sorting or conditioning role (e.g.. Bowles and 
Gintis 1976) or a role of securing a privileged 
position in the job queue (Thurow 1975). 
Human capital theory might be regarded as 
the basis for this approach, but we take the 
position that rational decisions in investing 
for desired returns are not sufficient to cap- 
ture the overall value and meaning of human 
existence. Thus, although our discussion will 
be in many respects consistent with human 
capital conceptions, we do not limit ourselves 
to expressing the values we have accepted 
nor the outcomes we discern as having nor- 
mative value, within the confines of that theo- 
retical perspective. In other words, we accept 
an altruism that values human existence in its 
own right, independent of economic 
interpretations. 

We also recognize that the institutions 
delivering formal education reflect the con- 
flicting forces and differences of opinion that 
permeate the society. They well may serve as 
z sorting mechanism, in part a function of 
their time-bound nature and the imperfect 
professionals who structure the experiences 
for students within them. The results of 
acquiring the dispositions may be interpreted 
by employers as proxies for the ability to 
learn new skills as yet unknown. As a conse- 
quence, noncredentialed workers may be 
displaced or not hired, even though they 
possess the needed skills or ability to learn. 
These conditions reflect the imperfections of 
the enterprise that we seek to understand in 
order to improve. We therefore cor)sider the 
outcomes as goals and the processes as 
ways to move closer to the goals. 



Modeling the Elements of 
Vocational Education Quality 

The proposed framework for the ele- 
ments of vocational education quality was 



the result of a multiphased process. A review 
of the literature, expert judgment, and key 
informant interviews provided input to the 
modeling process. 

The literature on effective school 
research and related subjects was the initial 
source considered in developing the model. 
Although the literature directly relating to 
and identifying the elements of vocational 
education quality was quite segmented, ref- 
erences were discovered that addressed vir- 
tually all the major components of our 
framework. The content of the references 
ranged from general observations and com- 
ments about effective programs to sugges- 
tions for specific indicators of quality. 
Related references were subsequently 
grouped and the resulting significant clusters 
were incorporated as model components or 
elements. 

To illustrate briefly the method by which 
references were utilized in model building, 
consider the following examples. Miller 
(1984) and McKinney et al. (1981) comment 
that effective schools are characterized by 
total commitment of the school unit to quality 
programs. This reference could be generally 
classified as an organizational characteristic 
of school setting. More specifically, it could 
be considered as an example of a general 
policy statement that is common to quality 
programs. References were also found relat- 
ing to organizational structure (Westbrook 
1982) and strategy (Benson 1982). This type 
of information was subsequently utilized to 
build a preliminary general framework of 
school setting that was structured as follows 

• Organizational Aspects 
—structure 
—policy 
—strategy 

As more information was gathered, a 
physical aspect of school setting was also 
identified, structured, and added to the 
model. The process was followed and even- 
tually resulted in a preliminary, systematic 
framework identifying the elements of voca- 
tional education quality 
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Experts in the field were involved in 
model definition through normative group 
proceetes, modified Delphi approaches, and 
individual review. Expert input was utilized to 
cross-validate components suggested by the 
literature, to suggest additional components 
to our model, and to develop more detailed 
definitions of components. 

The final source of information was key 
informant interviews. Nine administrators, six 
instructors/department heads, and six pro- 
gram graduates representing secondary and 
postsecondary vocational education pro- 
grams participated in an unstructured inter- 
view process, interviews were conducted as 
open discussions on the topic of "Quality 
Vocational Education.*' The role of the inter- 
viewers was to keep the discussion focused 
on the general topic and to provide prompts 
or suggestions for related discussion. 

In observance of Federal regulations 
regarding over-collection of information, the 
sample size was intentionsLlly kept small. 
Even though they were not technically repre- 
sentative of the population of interest, the 
interviews were valuable to the model- 
formulating process. Interviews were content 
analyzed and informally used to cross- 
validate the basic model suggested by expert 



judgment and the literature Interview infor- 
mation also aided in defining the 
components and elements of quality and in 
providing examples of potential indicators of 
quality 

In summary, our model was developed 
through a synthesis of the information 
gathered from experts, practitioners, and 
related literature The model is proposed as 
tr, attempt to define in an integrated fashion 
the people, processes, and structures that 
define the elements of vocational education 
quality 



Organization of This Publication 



The remainder of this publication is 
divided into three chapters. Chapter 2 
introduces and defines the elements of qual- 
ity related to educational outcomes that were 
u.::ntified through the previously described 
model-building process. Chapter 3 parallels 
chapter 2 in structure, but focuses on the 
elements of quality related to the educational 
experience. Finally, chapter 4 presents some 
issues and guidelines regarding the devel- 
opment of quality indicators for the elements 
within the proposed model 
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CHAPTER 2 



EDUCATIONAL OUTCOMES OR GOALS 



Two kinds of outcomes or go«.ls are sug- 
gested in figure 1: individual and societal, 
with a subset of societal outcomes specific to 
employers. They are not presented as 
exhaustive, but were selected to represent 
the key elements deemed necessary for a 
productive and satisfying life. 



Individual Outcomea 

Within the parameters of individual out- 
comes are several components. The first of 
these is called competency development. 
Aspects of this outcome include the devel* 
opment of work*related skills, both occupa* 
tional and job specific. They include the 
developmert of the basic skills suggested 
earlier in th s paper as being part of the 
general rigfit of individuals and obligation of 
education. They also include the develop- 
ment of general problem-solving ability. 

It should be understood that, as stated 
earlier, the development of the more recog- 
nizable vocational skills is not confined to 
formal education, either secondary or post- 
secondary; rather, both of these levels pro- 
vide avenues for their development. The 
processes (methods) currently employed to 
develop the necessary dispositions in the 
basic skills, again using Frankena's terms 
(1974). are primarily those of formal educa- 
tion. This premise is supported by law, by 
published opinion, and by traditional 
practice. 

Examples of the occupational and job- 
specific skills are such things as drafting, 
keyboarding, engine troubleshooting, pro- 
gramming, and crop planning. There are also 
more general, occupation- and job-related 
skills that include knowledge of the world of 



work, job search strategies, interview pro- 
tocol, and work conventions (arriving on 
time, following work rules). In most work 
situations, some ability to communicate 
using either spoken or written language is 
necessary. In many, the solution of quantita- 
tive problems is another significant element. 
The general ability to make appropriate 
judgments about the consequences of one's 
actions has major importance. Also, a recog- 
nition of fundamental relationships that are 
the stuff of science plays a part, whareas 
specific understanding of certain chemical, 
physical, biological, and economic laws fig- 
ures in the world of work in many instances 
For example, failure to understand the role of 
soil nutrients leaves the development of a 
crop to happenstance. Or, the likelihood of a 
business failure is increased by not under- 
standing that when costs exceed revenues, 
the enterprise is bound to fail when its avail- 
able capital is depleted to a critical level. 
Competency development is judged to be a 
key component because it provides the indi- 
vidual with a currency to exchange for 
needed goods and services, the necessary 
dispositions to accomplish self-services 
independently of other people, and the ability 
to add to the context of life in a positive way 
through contributions to society. 

Another individual competency with 
increasingly important potential might be 
identified as "information skills."* As the 
society changes trom an industrial base to an 
information base, a great variety of new 
sources and kinds of information become 
important— and as jobs change, these altera- 
tions frequently require accessing new 
information. The need for this skill is Implied 
within many of the basic skills, but it is 
worthy of emphasis as a separate and distinct 
skill because of its importance It includes 



'Thit conception was contnbuted by one of our reviewers Robert McCabe. President Miami-Dade Community College 
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finding, evaluating, applying, and communi- 
cating information. It will apply not only to 
stability and growth within a vocation, but 
also to other aspects of living as well. 

The second component, positive inter- 
personei interaction, permeates rot only the 
work role of individuals but also the total life 
space. Here individual fulfillment of nearly all 
people depends on having positive and mu- 
tually supportive relations with key people 
whose lives mesh with theirs in thp v^ork- 
place. the home, and the la ger community. 
To do this, they need to be able to communi- 
cate, to be sensitive to the impact that they 
have on their colleagues, and to develop and 
maintain a sense of shared responsibility. 

A wide range of interpersonal interac- 
tions might be labeled positive. At the one 
extreme are those people, some employers 
among them, who want docile response to 
requests and directives. At the other are 
those who demand individual assertiveness 
without regard for the consequences to oth- 
ers. We define positive, interpersonal actions 
as being neither of these extremes but rather 
a set of behaviors involving other people. 
These behaviors are adjusted to the circum- 
stances, but the individuals do not accept 
indefinitely repressive situations: they are 
responsive to the needs of others, but not 
insensitive to their own fulfillment. This point 
of view is analogous to Sternberg's (1984) 
contextual subtheory of adaptation, selec- 
tion, and shaping. According to Sternberg's 
theory, individuals first try to adapt to their 
environments. If this is impossible or undesir- 
able, they try to select a new environment. 
Should that not work out. they try to adapt 
the environment to make it a better fit. Posi- 
tive interpersonal relations involve activities 
of this kind. We do not assume, however, the 
order implied here We do assume that the 
disposition or behaviors required can be 
developed, rather than being inherent. 

A summary of research consistent witn a 
definition of this sort is also found in 
Tennyson et al. (1980). These authors con- 
sider both the work situation and other socal 



settings They identify the functionality of 
positive interpersonal relations, which they 
call "mature," in a variety of settings. In partic- 
ular, they imply that better problem solvmg 
will result from those mature relationships 
because less effort will be spent on defensive 
reactions and more on eliciting information 
and developing solutions. These constructive 
actions will lead to reduced conflict and less 
occurrence of defensive behavior. 

A third individual outcome that we postu- 
late is a sense of efficacy. Stated simply, it 
implies that individuals not only have some 
skills, but also sense their ability to use them 
to accomplish something. They expect to be 
able to get a job, to accomplish tasks deemed 
to be worthwhile, and to exercise a reason- 
able degree of control over their immediate 
personal environment. 

Beginning with the postulations of Kurt 
Lewin that the likelihood of undertaking a 
task is related to an individuaTs perception of 
the likelihood of success, the role of a sense 
of efficacy in motivation has been detailed by 
numerous researchers (For example, see 
Kelly 1967: Wong and Weiner 1981: Taylor 
1983.) 

The theories developed by these 
researchers and the evidence they have 
examined strongly suggest the importance of 
a sense of efficacy. In contrast, the concept 
of learned helplessness (Abramson. 
Seligman. and Teasdale 1978) and of its con- 
sequences illustrates the debilitating effect of 
the opposite pole of the efficacy continuum 
Also, there is empirical evidence that a sense 
of efficacy is associated with certain aspects 
of job satisfaction. For example, a recent 
study of job satisfaction, utilizing the 
National Longitudinal Survey Youth data 
(Campbell et al. 1982) found a consistent 
association between four different forms of 
job satisfaction and scores on a scale that 
included items such as "Tm able to do things 
as well as most other people." These exam- 
ples of theoretical and empirical evidence 
add to a normative and humanistic concern 
for the qualities of individual satisfaction to 
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support the mclusion of a sense of efficacy 
as a major individual outcome of formal 
education. 

The fourth major outcome for individuals 
includes to some degree the preceding three. 
It is designated as a sense of indBpetyd^nce. 
It means that the individual can manage a 
large array of ordinary tasks without having 
to depend upon someone else. It means that 
a man can provide a balanced meal when 
needed and that a woman is not helpless 
when confronted with a flat tire. It means that 
the individual chooses among alternatives 
rather than being channeled along single 
pathway of life episodes. 

A sense of independence has a strong 
normative background demonstrated in such 
diverse documents as the Declaration of 
Independence and the literature on therapeu- 
tic counseling. Its relationship to specific 
work life as well as its role in the general 
quality of life is suggested by a variety of evi- 
dence. It is reportedly related to job satisfac- 
tion among adults. For example, in a recent 
study. Hills and Crowley (1983) found that a 
sense of independence, operationally defined 
as a sense of autonomy on the job. was sig- 
nificantly related to a scale score of job satis- 
faction. This result is a further verification of 
Robert Blauner's classic study. Alienation 
andFrBBdom (1964). Katzell (1979) also pro- 
vides an analysis of the growing awareness 
of the legitimacy of this sense among the 
current work force. Although these 
relationships do not establish a sense of 
independence as a goal, they document its 
pervasiveness as a human valu< which, in 
our judgment, is the ultimate criterion for 
justifying it as a goal. (See also Taylor (1974] 
for a discussion of the logical and philo- 
sophical basis for this contention.) 

These four individual outcomes rest 
upon a class of intermediate outcomes that 
are not so much ends in themselves but are 
necessary in some form in order to attain the 
more fundamental results. Examples are the 
development of self-helping skills (tire chang- 
ing, cooking, keyboarding). information 



about alternatives (career awareness), and 
the broadening of each individual's repertoire 
of knowledge (enrichment). Vocational edu- 
cation functions with respect to these out- 
comes in varied ways. In some cases, it may 
be the sole, school-based source of oppor- 
tunity to achieve the outcomfj (cooking) In 
others, achieving the outcome may be shared 
with alternative curricula (general, academic) 
in the formal school setting. 



Societal Outcomes 

The societal outcomes are five in 
number. In order of generality, social stabthty 
is probably highest. It is necessary for the 
other intermediate outcomes to be possible, 
but it also depends, at least in part, upon 
their attainment. By social stability we do not 
mean social stagnation. Rather, we mean 
gradual, orderly, and nonviolent change, with 
expectations within the society being rea- 
sonably predictable, at least for a long 
enough period that changes may be planned, 
rather than being unpredictable shocks. 
Neither education in general, nor certainly 
vocuiional education in particular, can 
accomplish this outcome alone, but the expe- 
riences that occur in vocational education 
s hould meet the test of a positive contribu- 
tion to social stability. 

The remaining four societal outcomes 
are much more specific. They represent prod- 
ucts that can be addressed quite directly by 
vocational education. 

Providing essential services to the 
society is an obvious need. Trained people m 
health care, information processing, and the 
highly skilled components of durable manu- 
facturing are needed for a smoothly function- 
ing, modern society. It is recognized that not 
all essential services are provided through 
the labor market or through paid work The 
quality of a society also rests heavily upon 
those services that are provided m the home 
and by volunteers in various roles. 
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Although it is true that services viewed as 
essential by the majority of the society are 
probably available through some form of 
enterprise that becomes part of the labor 
market, not all services are so provided. 
Many are available both on a paid basis and 
on a donated basis. Numerous examples 
exist of the latter, all of which contribute to 
the quality of life in a society but do not 
require an exchange of money, goods, or 
services. Volunteers often provide some form 
of nursing care to sick patients, technical 
advice to amateurs on home repair free 
assistance on car repairs, free garden pro- 
duce, snow shoveling, attentive concern to 
people in trouble, family management, par- 
enting, teaching (e.g. church school, informal 
music lessons), advice on nutrition, politics, 
and ways to cope with a complex society. 
Whether government policies and profes- 
sionals in these areas concur or not. these 
volunteer activities do take place and will 
continue to do so. Education, both formal, 
general, and vocational, becomes one of the 
mechanisms through which volunteers can 
learn appropriate skills and roles and can 
therefore contribute to the well-being of the 
society. 

Establishing a scale of values for out- 
comes of this sort is not easy. This difficulty, 
however, is a very inadequate reason for 
ignoring the contribution made by the non- 
labor market elements of the society. The 
presence or absence of skills, and reports of 
engagement in nonpaid volunteer activities, 
may provide a gross estimate. Some attempts 
have been made to place dollar values on 
such services (e.g.. the value of homemak- 
ing) but to a very large degree the problem is 
unresolved. Nevertheless, the contribution is 
too great to ignore. 

Closely related to the essential services 
is the outcome termed quality of work prod- 
ucts. It extends beyond the tangible and 
material quality of things to the quality of 
services provided. A disillusionment with 
quality in this broad sense leads to an ero- 
sion of stability and a reduction in the overall 
quality of life for the society as a whole 



We are not aware of a body of literature 
that specifically documents this assertion, 
but several exan^ples will illustrate the intui- 
tive logic of this position First, consider the 
current plight of the automobile industry. 
Beginning in approximately 1974. the declin- 
ing demand for American automobiles 
became increasingly recognized. One result 
was that the market for replacement automo- 
biles began to decline as consumers chose to 
retain the vehicles they owned rather than 
pay higher prices for less satisfactory vehi- 
cles This, in turn, reduced the effectiveness 
of pollution control devices because non- 
equipped automobiles stayed on the road 
longer. It also reduced the contribution of the 
auto industry to the economy and was a large 
factor in the turn toward imported autos. 
which further eroded the economy within the 
United States. Ten years later the conse- 
quences are still strongly felt. 

A second example applies to a service 
area Suppose a child from a low socioeco- 
nomic status family is having serious achieve- 
ment troubie in school The mother visits the 
child s teacher to see what can be done. Both 
the mother and the teacher are sincerely 
interested in the child's welfare. But the 
teacher does not know how to empathize and 
communicate with the mother, and the 
mother is made to feel inferior and inade- 
quate. She t^erefore avoids continuing 
school visits, and. as a result, the real contri- 
bution she could make is lost The child's 
education continues to suffer. 

A third example, in an entirely different 
area, concerns a hypothetical situation in the 
delivery of medical care. A patient goes to a 
physician's office with an annoying but not 
too painful ailment, as yet undiagnosed. The 
patient is treated with perfunctory courtesy 
More attention is given to the ability to pay 
than to diagnosis of the ailment, and the phy- 
sician appears to be hurried and not very 
interested. The next time the symptoms 
appear, the patient puts off seeking medical 
service until a serious and perhaps irreversi- 
ble condition develops. 



We have emphasized in these exemples 
the service products more than the tangible 
products. Deterioration in the quality of 
either contributes to the disruption of society 
through disillusion with its institutions, 
through dispiecement of people (witness the 
Michigan-Ariiona migration phenomenon as 
dispteced auto workers sought jobs in the 
Southwest), and through more extreme dis- 
ruptions such as riots, hijackings, and the 
organizing of private armies. 

These results are. of course, not attribut- 
able to lowered product quality slone. but 
such a phenomenon increases the potential 
for serious societal consequences. 

The third outcome suggested in figure 1 
as relating to social stability is the accumula- 
tion of the necessary resources for se/f- 
tuppoft by the substantial majority of the 
members of a society. The proportion of the 
society engaged in productive work that may 
t>e exchanged for other goods and services is 
a function of t>oth the standard of living and 
the productivity of the society. 

If individual productivity is relatively low 
and the desired standard of living is high, 
then proportionately more persons will need 
to be employed in a self-supporting way in 
order to carry the f emaining members of the 
society at a similar level. Conversely, a 
smaller proportion of more highly productive 
self-supporting people can carry the remain- 
ing members assuming the standard of living 
stays constant. If. however, the standard of 
living is lower, less individual productivity is 
r^essary to maintain the living standard. In 
immediate and practical terms, the actuarial 
difficulties of the Social Security system illus- 
trate this problem. As the proportion of 
retired <^ >rkers who are no longer productive 
rise;» in relation to those paying Social 
Security taxes, the Nation is forced to choosd 
some combination of four alternatives: 

• Accept a lower standard of living due 
to higher taxes and inflation 



• Impose a lower standard of living on 
the retired group by maintaining a 
fixed income level despite inflation 

• Increase sharply the productivity of 
the working group 

• Encourage continued productivity of 
the retired group 

4 is again difficult to specify just how 
progress toward this goal should be mea* 
sured. However, a reasonable goal to which 
vocational education should contribute is the 
increase or maintenance of such a productive 
majority. 

A society in which every individual con- 
tributes to his or her own betterment also 
contributes to the betterment of the lives of 
others in that society. This is what is meant 
by sociera/Zmproi^emenr— working as indi- 
viduals to improve the quality of life for the 
aggregate. This type of positive growth is 
considered necessary in order for a society 
to sustain itself rather than stagnate. Thus, 
societal improvement the final critical ele- 
ment supporting serial stability. 



Societal Outcomes Specific to Employers 

A subset of the societal outcomes are 
those that impact employers directly. To be 
worth the investment from the employer s 
point of view, vocational education should 
contribute to ensuring that needed skills are 
available, that on-the-job training time for 
graduates of vocational programs is less than 
for those without such training, and that the 
productivity of these trained workers is 
higher than that of untrained workers. To 
date, the evaluation of vocational education 
has been largely limited to factors such as 
these, while ignoring the numerous other 
outcomes suggested in figure 1. It is our 
judgment that although these outcomes are 
necessary in the paid work market, they are 
not sufficient for assessing the quality of 
vocational education there or in the other 
defined outcome areas. 
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Th% Plact of Vocational Education 

We began this discussion by considering 
the role of vocational education as a part of 
the broad concept of education m general 
We have not been willing to accept a narrow 
definition that divorces the training in spe* 
cific, employment-related skills from the 
education of a person for all aspects of living 
Thus, our definition of vocational education 
includes, but is not limited to, employment- 
related Skills. Development of those skills is a 
necessary, though not sufficient, component 
of vocational education as we see it. This 
leads to a recognition that vocational educa- 
tion of a positive quality must develop 
employment-related skills; it may not sub^ 
stantialiy interfere with the development of 
desirable or even essential skills or disposi- 
tions that are not directly employment 
related; and it should, to the extent possible, 
enhance those dispositions. (For a succinct, 
normative discussion of some of these dispo- 
sitions, see Thimmesh 1984 ) 



At the secondary level, vocational educa- 
tion competes for class time with other elec- 
tives and with required courses that reflect 
American society's collective judgment about 
what is basic and essential. At the postsec- 
ondary level, a similar competition is present 
in the planned sequences but is less enforced 
because of the greater legitimacy of taking 
one or two courses for immediate personal 
goals and then discontinuing the program * 
These conditions require that vocational 
education share responsibility for the kinds 
of goals shown in figure 1, although it cannot 
be held responsible for their complete 
attainment, nor can the larger subset of for- 
mal education. It logically follows that the 
quality of vocational education can only be 
assessed by taking into account its role in the 
larger context. 



•In addition some people assunr>e that some of the goats have already been attained by the time of secondary school com- 
pletion and consequently need not be readdressed at the p03tsecond&ry lev?) 
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CHAPTER 3 



THE EDUCATIONAL EXPERIENCE 



GtMral Otfinltlofi of tht 
Educational Eiporionct 

The vocational educational experience 
involves a network of interrelated and inter- 
dependent people, processes, and structures. 
A complex environment comprising a 
number of interacting parts, it functions 
within the general education system and 
within a much larger environmental context. 
In order to deterrrine whether the environ- 
ment is operating effectively in achieving 
desired individual and societal outcomes and 
in order to make improvements in the educa- 
tional experience as a whole, it is essential to 
understand how each part functions, both 
independently and interdependently. Thus, 
defining in detail the elements of quality in 
the educational experience is a critical step 
toward achieving these ends. 

The conceptual framework proposed in 
figure 1 provides a starting point for discus- 
sion. The educational experience repre- 
sented by the central panel of the model is 
situated within the larger context of environ- 
mental inputs and outcomes shown in the 
left-hand and right-hand panels. The central 
panel is divided into "School Context," dis- 
played in the uppermost cell, and "School 
Process/' represented by the three lower 
cells of the panel. The components iisied 
within all four cells define the boundaries of 
the school context and school process 
domains. 



Scliool Context 

School context deals with both organiza- 
tional and physical aspects of the school 
unit. It is represented in figure 1 as the first 
cell in the central panel. Personnel, facilities. 



equipment, policies, and formal organiza- 
tional structure are general elements of 
school context. 

School context sets the stage for school 
process and can best be described through 
illustration. The following two fictitious "con- 
text** scenarios help to show the place of 
school context within the overall educational 
experience. 

The development Middletown Techni- 
cal Institute (MTI), a postsecondary voca- 
tional education school, was a direct 
outgrowth of current and projected needs of 
business and industry in Middletown. U.S.A. 
Goals of the school and its individual pro- 
grams were developed by a committee that 
included educators, administrators, students, 
parents, and representatives of business and 
industry. Consideration of school and pro- 
gram goals influenced the design of the phys- 
ical structure of the school as well as the 
purchase of equipment. Local business and 
industry demonstrated support of MTI 
through generous donations of equipment 
and material. Selection criteria for instructors 
included technical experience and knowl- 
edge, formal training in education methods, 
enthusiasm for teaching, and other selected 
variables. An active business-industry advi- 
sory board helped MTI stay abreast of chang- 
ing needs. Qualified counselors were 
charged with tailoring student programs to 
individual needs and assisting in job-seeking 
efforts. MTI enjoyed a favorable image in the 
community. There were more applicants to 
MTI than its programs could handle; student 
selection criteria were established to manage 
the demand overload. 

Steeltown Vocational School (SVS). our 
second fictitious program, presents a much 
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difftrent picture of school context. SVS is a 
postsecondary institution that was opened in 
the earty 19SOs to train skilled laborers for 
the local steel and machine tool industries. 
School and program goals have not been 
modified since its inception. The school 
buifding is a converted warehouse and the 
instructional staff is mainly composed of 
former supervisors of sicilled laborers. 
Instructors have litde or no formal education 
in teaching methods. The physical facility is 
in desperate need of repair having craciced 
walls and ceilings and antiquated plumbing, 
heating, and lighting systems. However, with 
enrollment and revenues down, there are no 
plans for malcing the needed repairs. Place- 
ment rates for SVS graduates are at en all- 
time low. Support from the community is 
virtually nonexistent; it is only a matter of 
time before SVS will be closed. 

These scenarios help to illustrate the 
importance of school context in "setting the 
stage*' for the educational experience. Per- 
sonnel, facilities, equipment, and organiza- 
tional structure are all part of the "context" of 
an educational facility and relate to the qual- 
ity of a facility. The contrast between the 
school context of MTI and SVS was exagger- 
ated in order to demonstrate the significance 
of context in the educational experience. A 
more detailed discussion of the components 
and elements of school context will appear 
later in this chapter. 



School Process 

If school context provides the setting, 
school process provides the action. Referring 
again to figure 1 , school process is repre- 
sented by the second, third, and fourth cells 
of the central panel. Cell 2. entitled ^'Student 
Utilization.*' relates to pM0rn% of participa- 
tion that are influenced by school context 
and school process (Campbell. Gardner, and 
Seitz 1982). Cell 3. "Teacher. Student. Mate- 
rial Interaction." combined with Cell 4. 
"Instructional Content and Equipment." 
defines the teaching-learning process. 



Descriptions of school process include 
references to course content, course-specific 
equipment, and materials. Discussions of 
student-teacher, student-student, student- 
material interface, and level of formal student 
involvement in the vocational education 
school unit or program are also involved in a 
comprehensive description and analysis of 
school process quality. 

The following description of a fictitious, 
comprehensive high school may help to illus- 
trate the general boundaries of the school 
process domain. 

Jefferson Comprehensive High School is 
viewed mainly as a college preparatory insti- 
tution. Students are discouraged from for- 
mally entering the vocational education 
program, so those students that take voca- 
tional education courseworic do so mainly as 
8 diversion from purely academic course 
loads. Concentrators are generally held in 
low esteem by their peers and by the majority 
of academic program instructional staff. 
Vocational education instructors have little 
influence on purchase of instructional 
equipment and are forced to use out-dated 
textbool(sand equipment. Students have 
limited opportunity for *'hands-on" expe- 
rience These restrictions and limitations are 
also reflected in episodes of student-teacher, 
student-student, and student-material inter- 
action. Instruction in vocational education 
courses appears to be a one-way process. 
Teachers dispense information and students 
absorb information; a feedbacic loop is virtu- 
ally nonexistent between teacher and stu* 
dents Typically, student-student interface 
consists of rubber band shooting, note pass- 
ing, and joice telling Because of the \ack of 
support and resources for vocational educa- 
tion programs, there is a high rate of turnover 
of 'good' vocational education teachers. It 
seems that those who stay are the type who 
care little for the job and the well-being of the 
student. 

This description touches upon the major 
components of quality in the school process 
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portion of the model. It provides a general 
**feer* for the $chool process domain and the 
potential effect of process on the overall 
educational experience. 



IntenelatlofitMp between the 
Edueatioiiel Experience and Outcome 



The processes and structures that define 
the elements of quality within the vocational 
education experience should relate to 
intended individual and societal outcomes 
The educational experience should increase 
the likelihood that desired individual and 
societal outcomes are achieved. In order to 
do so, a systematic understanding is required 
of the relationships between components of 
the educational experience and outcomes 
This will allow the school unit to maintain 
and replicate effective programs and will 
promote systematic program modification as 
outcome indicators change. 

For example, with the current emphasis 
on and movement toward electronic data 
processing both at home and on the job. 
computer literacy may eventually be included 
along with basic math. English, and so forth 
as part of basic skills. In turn, the addition of 
computer literacy should be reflected at the 
outcome indicator level and also in the form 
of school context and school process indica- 
tors. In other words, there.should be a sys* 
tematic linkage between desired outcome 
and experience; the educational experience 
should be responsive to changes in desired 
outcomes. 

In order for the linkage to be systematic, 
the educational experience must be dis- 
sected into observable components or ele- 
ments of quality that relate to intermediate 
objectives. Achievement of the se objectives 
should support the achievement of desired 
outcomes. Thus, outcomes are considered as 
goals, and the educational experience as the 
way to move closer to the goals. 



Elements of School Context 

A number of components related to qual- 
ity of education are classified within the 
school context and process domains of our 
model. For example, as figure 1 indicates, the 
components of school context are these: 

• School setting 

• Classroom setting 

• Teacher skills 

• Teacher characteristics 

• Peer group characteristics 

Each component consists of a set of ele- 
ments defining its parameters. A thorough 
understanding of elements of quality is pre- 
requisite to developing specific indicators of 
quality. Therefore, the elements of quality 
related to school context anu school process 
will now t>e identified and discussed in detail. 



School Setting 

Our model proposes five components of 
quality within the school context domain (see 
f gure 2). The first of these is school setting 
School setting encompasses both physical 
and organizational aspects of the educational 
experience. The physical elements of school 
setting include the school building, the 
school property, and general equipment as 
opposed to program-specific equipment. 
Integrity of basic systems such as structural 
support systems, plumbing and heating sys- 
tems, ventilation and lighting systems, and 
exterior and interior design features would be 
considered as part of the "school building." 
The second element, school property, deals 
with the adequacy of adjunct facilities, such 
as parking lots. The category of general sup- 
port equipment is the third physical element 
related to quality of school setting. General 
support equipment is defined as equipment 
related to the functioning of the school unit 
as a whole and does not include equipment 
purchased for individual programs. Copy 
machines, telephones, file cabinets, and 
management related hardware and software 
are examples of general support equipment. 
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Figure 2. Elements of quality of school context 
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A lathe is considered program-specific 
equipment and would not be included here. 

Elements within the organizational 
aspect of school setting include formal 
organixationai structure, strategies, and poli- 
cies. Organizational structure relates to the 
formal organizational chart and related man- 
agement philosophy/ It plays a major role in 
defining school setting since it directly 
affects personnel networks and information 
flow. For instance, one school unit might 
have a rigid, hierarchical organization chart 
with an autocratic management approach, a 
top*down flow of information, and a principal 
who is primarily concerned with manage- 
ment. A second school might also have a 
hierarchical structure but one that is defined 
by a participative management style (e.g.. 
collegiality at the community college level), 
an interactive communication network, and 
an administrator who is primarily concerned 
with instruction. 

Strategies relate to long-range goals and 
directions of the school as a unit. Research 
indicates that schools with clearly stated 
mission statements are more effective (AASA 
1983). The basic strategic alternative of 
growth vs. no-growth is an example of a stra- 
tegic direction that would affect school set- 
ting. An example of a strategic goal would be 
minimization of capital expenditures for 
equipment through development of and 
emphasis on a strong industry donation pro- 
gram. This goal could potentially affect the 
activities and responsibilities of administra- 
tors, instructors, and students and would 
therefore need to be considered in a descrip- 
tion of school setting. 

Organizational policy is the final element 
within the organizational aspects of school 
setting. Policy affects the daily operation of 
the organizaton. Dress code is one example 
of a policy decision. Or. consider a 2-year 
college policy regarding credit for nontradi- 
tionai learning experiences. 

Thorough consideration of the physical 
and organi;eational aspects of school setting 



should result in a comprehensive description 
of the subelements of quality within that 
domain. Only then can judgments be made of 
adequacy, appropriateness, and effectiveness 
and indicators of quality be developed. 



Classroom Setting 

Classroom setting is the second compo- 
nent of school context. It is much narrower in 
focus than school setting. Classroom setting 
deals with physical aspects of the classroom 
as they relate to general instruction. Class- 
room setting deals with the overall physical 
comfort, safety, and adequacy of the general 
instructional sett .j (Westbrook 1982). 
Aspects to be considered would include ade- 
quacy of safety equipment; adequacy of light- 
ing, ventitation and heating: adequacy of 
desks and work areas; and availability of 
basic teaching equipment such as black- 
boards, pointers, flip charts, and more tech- 
nologically advanced teaching equipment 
such as videotapes, computers, closed circuit 
TVs, and so forth. Classroom setting is of 
particular importance in a vocational educa- 
tion setting. For example, in a course such as 
c welding lab where open flames are neces- 
sary, ventilation and traffic flow patterns are 
of extreme importance. Availability and ade- 
quacy of safety equipment and procedures 
are also critical. Elements of classroom set- 
ting are important to general instructional 
settings too. Most of us know what is it like to 
take a test in a cold, poorly lit classroom so 
we can understand why classroom setting 
has an effect on the overall educational 
experience. The underlying premise is that 
certain standards should be met in order to 
enhance learning and increase the probabil- 
ity of meeting course objectives. 



Teacher Skills 

The third component of school context is 
teacher skills. Three basic elements of quality 
are included in our definition of teacher 
skills. First is technical experience— l^ow 
much direct work experience do teachers 



*A6miniStrator skills and characteristics are subsumed within the elements of the organizational aspect of school setting 
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have in the fields in which they are teaching? 
Second is teaching methods, knowledge, and 
skills— Has the teacher completed postsec- 
ondary course work in history of education, 
teaching methods, approaches, and tech- 
niques? Does the teacher apply this knowl- 
edge to the classroom setting? Does the 
teacher present subject matter in a form or 
manner that makes it more meaningful and 
significant to the learner (National Commis- 
sion on Secondary Vocational Education 
1984)? Third is technical knowledge— Is the 
teacher technologically up-to-date (National 
Research Council 1983)? Has the teacher 
continued formal course work related to the 
teaching field? Does the teacher have recent 
field experience? These are the elements to 
be considered and some general questions 
addressed when assessing the teacher skills 
component of school context. 



Teacher Characteristics 

The fourth component of school context 
is teacher characteristics, defined by two 
quality-related elements: teaching approach 
and teaching style. Teacher characteristics is 
a much "softer" component than teacher 
skills, but it is no less important in terms of 
its effect on the educational experience. 

Teaching approach is related to teaching 
philosophy and is manifested as attitude 
toward the teaching-learning process as 
demonstrated through daily behavior both in 
and out of the classroom. Is learning a one- 
way process with the teacher dispensing 
information and the student absorbing infor- 
mation, or is the student-teacher feedback 
loop a critical element of process? Is learning 
to be dreaded or enjoyed? Does learning 
have a definite start and finish or is it a life- 
long process? Are the teacher and student on 
the same team or opposing teams? The 
answers to these types of questions help to 
define teaching approach. 

Teaching style is the result of a number 
of factors. Attitude toward job. attitude 
toward students, subject and content area. 



and individual personality all affect teaching 
style Teaching style is exoressed through 
teacher classroom behavior Is the teacher 
enthusiastic and innovative? Is humor incor- 
porated, as appropriate, into the teaching 
process? Does the teacher show empathy 
with students? Is the teacher concerned with 
individual students? The teacher behaviors 
that provide the answers to these and similar 
types of questions constitute teaching style 

In summary, the teacher characteristics 
category is the combination of a range of var- 
iables falling within the elements of style and 
approach. There is undoubtedly a range of 
effective teaching styles and approaches. 
However, in order to develop indicators of 
quality teacher characteristics, the assump- 
tion must be made that there are identifiable 
characteristics that are common to effective 
styles and approaches. Most of us have 
memories of specific teachers whose styles 
and approaches were either particularly 
«^ffective or ineffective. One can therefore 
easily understand why teacher characteris- 
tics play an important part of the overall 
learning experience. 



Peer Group Characteristics 

The fifth and final component of school 
context is peer group characteristics. This 
can generally be defined as the attitude 
toward education, academic performance, 
and behavior of the student body as a whole 
or of a homogeneous group within the stu- 
dent body For ex'^mple. secondary educa- 
tion could be synonymous with college 
preparatory education for one student body 
Another student group may view high school 
as a setting for courtship and socialization, a 
third may view secondary education as a 
stepping stone to the world of work, or a 
fourth group might view it as a setting for 
organized amateur athletic competition This 
aggregate view pervades the group and typi- 
cally affects students' behavior and perfor- 
mance at the individual level Peer pressure 
plays a significant part in the formulation and 
maintenance of peer group characteristics 
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Consider the following example. A homo* 
genous group of very bright students decides 
that "grades** are meiningless symbols. 
They claim that course content is not chal- 
lenging and "anyone can get an A." There- 
tore, grades do not differentiate the bright 
students from the marginal students. This 
peer group holds the learning process and 
school programs in very low esteem. They 
decide to express this attitude through poor 
attendance records, nonparticipatory behav- 
ior in the classroom, and failing grades in all 
their courses. In doing so, they are making a 
group statement about the educational expe- 
rience that affects group members' academic 
performance and behavior at the individual 
level. 

Thus, peer group characteristics may 
significantly affect the student*learning inter- 
face and can be perpetuated or modified 
through peer pressure systems. An under- 
standing of peer group characteristics is pra- 
requisite to a comprehensive definition of 
school context. 

School context "sets the stage" for 
school process, which will provide the 
"action. * Context relates to the quality of the 
physical and organizational setting through 
the school sstting and classroom seWng 
components and provides characterization of 
the main actors through the components of 
tssch§r skills, teacher charactarisUcs, pear 
group charactaristics, and school satting 
since it includes administrative skills and 
characteristics. 



Elements of School Process 

School process is the remaining portion 
of our conceptualization of elements of qual- 
ity in the educational experience. The 
taaching-laarning process and patterns of 
participation are its major subdivisions. 
Components and elements were identified 
and defined as part of the model formulation 
process described in chapter 1 . The results of 
that effort as they relate to school process 
follow. 



Teachlng^Leamlng Process 

The teaching-learning process includes 
two components related to quality: teacher, 
student, and material interaction and instruc- 
tional content and equipment. These compo- 
nents are represented as cells three and four 
of the central panel in our conceptual frame- 
work (figure 1) and are further detailed in 
figure 3. 

Instructional content and equipment. 

Instructional content and equipment deals 
with courso*specific information and equip- 
ment. Even though there is a wide range of 
course offerings within any given school unit, 
there are certain common elements related to 
quality of content and equipment that should 
be considered regardless of the course. 
These elements include but are not limited to 
the following: 

• Clear course objectives (Westbrook 
1982) 

• Business- and industry- related 
course content (National Research 
Council 1983; Benson 1982) 

• Instructional content related to 
course objectives (Westbrook 1982; 
Miller 1984) 

• Course objectives consistent with 
program goals (Westbrook 1982; 
Miller 1984) 

• Appropriate equipment available 

• Utilization of multiple media for 
instruction 

• Balance between materials requiring 
content knowledge, understanding, 
and application (Benson 1982) 

• Balance between knowledge, skill, 
and attitude development (National 
Research Council 1983) 

These elements and others like them 
serve as a starting point for the development 
of indicators of quality content and equip- 
ment. The underlying premise is that certain 
standards indicative of quality can be identi- 
fied for each element. 
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Student Utilization of 
Vocational Education 



Patterns of Participation 

• Concentrstors 

• Limited Concentrators 
e Concentrstor/Explorers 
e Explorers 

• Incidental/Personals 



T««clMK. ttudtnt. mattrial Inttractlon. 

The second component of the teaching/ 
learning procesa is labeled feacher. student 
material interaction. This component can 
generally be described in terms of five de- 
scriptive elements: type, class, frequency, 
duration, and nature. 

• Type refers to the specific, interactive 
combination (e.g., student with stu- 
dent, student with teacher, etc.). 

• Class refers to the basic classification 
of the interaction. Examples 
include— 

—teacher monitors student, 
—teacher questions student, 
—teacher reinforces student, 
—student reads material, 
—student handles equipment, 
—student questions teacher. 

• Frequency involves &n actual count 
of interactions by type or class. For 
instance, the number of student-with- 
teacher interactions occurring within 
a class period might be noted. 

• Duration involves a time measure of 
the length of each interaction. This is 
usually recorded by type of interac- 
tion (e.g.. How long did the student 
with student interface last?). 

• Nature involves a general assessment 
of the quality of the interface, such as 
positive, neutral, or negative. 

A considerable number of references 
aided in the development of this classifica- 
tion scheme. For example. Halasz. Behm. and 
Fisch (1984). Miller (1984). and Westbrook 
(1982) suggested an increase of "time on 
task" in effective programs. This would 
involve development of indicators that 
include type, class, and duration elements. 
Hunt and Woods (1979) recommended fre- 
quent positive reinforcement of student by 
teacher that would include the elements of 
type, class, and nature. Westbrook (1982) 



and Miller (1984) suggested that teachers 
should monitor student performance and 
provide qualitative feedback. This sort of 
interaction would be described by a combi- 
nation of all five elements in our scheme. 
Therefore, references helped to identify ele- 
ments related to quality of the teacher, sfu- 
denf, material interaction component. 

Thus, a description of student, teacher, 
and material interaction in the classroom 
would include a statement of type, class, fre- 
quency, duration, and nature. Development 
of indicators of quality for these interactions 
would then be made in consideration of 
instructional content and equipment. For 
instance, it may be determined that a lab set- 
ting requires a greater frequency and dura- 
tion of student-with'Student interaction than 
does a lecture class. Or, there may be a 
higher need for teachers to monitor individ- 
ual students in a lab setting than in a class- 
room setting. 

Therefore, a comprehensive analysis of 
the quality of the teaching-learning process 
component would include references to both 
instructional content and equipment and 
teacher, student, material interaction. This 
process provides much of the "action'* in the 
school unit and is clearly a major part of the 
overall quality of the educational experience. 

Student utilization of Inatructionat pro- 
cess. The balance of school process deals 
with the formal relationship between the stu- 
dent and the vocational education school 
unit or program. Campbell et al. (1982) iden- 
tified this interface as patterns of participa- 
tion: it has been incorporated into our model 
as a component of school process (figure 3). 
but it is also accurately described as an 
intermediate outcome of the interaction 
between school context and school process 
(figure 1). 

Campbell conceptualized a five- 
dtmensional matrix for describing possible 
patterns of participation. The dimensions of 
the matrix were intensity, diversity, conti- 
nuity, supportive diversity, and proximity 



21 



ERIC 



33 



Five patttmt wtrt hypothtsized and empiri- 
eaily MrifM. ThtM patterns ware identified 
by Campbell, Gardner, and Seitz (1M2) as 
foliowa: 

• Concentrators 

• Limited Concentrators 

• Concentrator/Explorers 

• Explorers 

• Incidental/Personals 

Briefly, lt>e descriptive concepta include: 
(1) the number of credits receive in 
vocational courses in the program area 
of specialization; (2) the number of pro- 
gram areas in which thr specialty was 
pursued; (3) the number of vocational 
credits in the program area that were 
delermin vi to be supportive of the spe- 
cialty area; (4) a scaled measure of 
whether the specialty was pursued in the 
11th or 12th grade; and (5) the number of 
years in which the specialty was taken. A 
student's area of specialization was 
defined as a program area (for example, 
distributive education, home economics) 
in which at least six-tenths of the total 
number of vocational credits v^ere 
received. 

For example. Concentrators take an 
average of 6 vocational credits over a 3- 
year period. Limited Concentrators 
generally take about half the number of 
vocational credits that Concenirators 
take, usually within a 2-year period. 
Concentrator/Explorer is similar to the 
Limited Concentrator pattern, except 
that the vocational course work is usually 
completed early in the high school yearu. 



Students classified in the Explorer pat- 
terns pursue courses in three or more 
program areas but do not achieve any 
level of specialization. In comparison. 
Incidental/Personal students average 
less than a full credit and generally com- 
plete the work in a semester. These pat- 
terns were used in the analyses in place 
of the traditional curriculum descriptors 
of vocational, general, and college 
preparatory, (p. 16) 

Although detailed discussion of these 
patterns goes beyond the scope of this 
report, it is important to be aware that the 
extent and kind of participation in vocational 
education is critical to a comprehensive eval- 
uation of quality of the educational expe- 
rience. Patterns of participation seem to be 
influenced by a numt>er of aspects of the 
educational experience, such as program 
structure, use of facilities and equipment, 
assignments of students, course objectives, 
and elements of teacher, student, and mate- 
rial interface. In other words, patterns of 
participation are interrelated with the 
teaching-learning process and school con- 
text and may therefore be described as an 
intermediate outcome of their interaction. 

Our conceptualization of elements of 
quality in the educational experience pro- 
vides a framework for developing indicators 
of quality. The model may not be compre- 
hensive, but a continuation of the process by 
which it was developed should lead to a 
complete definition of the concept. A general 
discussion regarding development of indica- 
tors of quality will be presented in chapter 4. 
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CHAPTER 4 



ASSESSING QUALITY OF VOCATIONAL EDUCATION 



Introduction 

It it no longer sufficient to My Johnny 
can read. One must also state fiow well he 
can read, why he can read, and what can be 
done to improve Johnny's reading ability. 
Consequently, evaluation has become a 
maior activity within the field of education 
today. The individual. Federal, State, and 
local agencies and society as a whole have 
concerns about the quality of educational 
processes, structures, and outcomes. There- 
fore, educators are being asked to assess a 
complex educational system by means of a 
complex evaluation system that is quite a 
complex undertaking in itself. 

Evaluation is a major area of emphasis in 
the Carl D. Perkins Vocational Education Act 
of 1084. In addition to the requirement for 
placennent and employment data for pro* 
grams assisted under the act, each State is 
required to evaluate program effectiveness 
and "quality and availability of instructional 
offerings." Consequently, within vocational 
education today, the evaluation of quality is 
emphasized through its ties to legislation and 
funding. 

Chapter 4 presents some general. * -M 
important, issues and guidelines related to 
assessing quality of vocational education. It 
is not intended to be a comprehensive review 
of program evaluation techniques, models, 
and methods; the literature on the topic is 
voluminous and could only be superficially 
treated within the scope of this writing. 
Rather, general evaluation approaches, spe- 
cific models, and so forth are discussed as 
they relate to developing indicators for the 
elements of quality vocational education that 
were discussed in chapters 2 and 3. 



What Is Quality? 

Chapters 1. 2. and 3 presented within a 
systems framework the components and 
elements that affect quality of vocational 
education. These components and elements 
are generalizable across school units and 
programs. But the specific indicators of qual- 
ity associated with them may vary with 
school unit and program and are likely to 
change over time. This is because specific 
definitions of what indicates quality are rela- 
tive, judgmental, and subject to change as 
the scope of the individual provider respon- 
sibility changes. 

The definition of quality education is 
strongly influenced by conceptual 
approaches operant within the field at a par- 
ticular point in time. Therefore, the definition 
of **good** depends on present knowledge 
and opinion. For example, in the early 1970s, 
the "open * classroom approach was adopted 
by many educators. Programs were modified 
and facilities were tailored to this new 
approach with the underlying assumption 
that the *'open" classroom was positively 
related to "quality** education. 

Consequently, although the components 
and elements of school context, school pro- 
cess, and particularly the educational out- 
comes remained intact, specific standards of 
quality related to those components probably 
changed. Consider, for example, classroom 
setting. One indicator of a quality classroom 
setting prior to the open classroom approach 
might have been this: 

• A closed, well-sound-proofed instruc- 
tional setting allowing X square feet 
per student and one desk per 
student. 
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For those schools that subsequently 
accepted the open classroom concept the 
parallel indicator would probably have been 
this: 

• An open instructional setting with no 
structural barriers between four 
independent classes and *work tables* 
designed for simultaneous use by 
four to six students (no individual 
desks for students or teachers). 

Of course, this is just one example of how a 
specific indicator of quality might change as 
approaches within the field change. In actual- 
ity, the open classroom approach undoubt- 
edly affected indicators associated with most 
of the elements of the school context and 
school process portion of our framework. 

The definition of quality also changes as 
scope of provider responsibility changes. 
Consider the growing demand for and role 
of day-care services today. Many children 
from the t*me they are infants are placed with 
day-care providers for 45 hours per week. 
These providers spend considerably more 
waking hours with these children than do 
their parents. The scope of the providers' role 
has potentially expanded from one of basic 
care provider to that of primary role model. 
The overall societal and individual assess- 
ment of the importance or substance of child 
raising has not changed. However, the defini- 
tion of day-care has expanded and the 
indicators of quality day-care should theoret- 
ically reflect this change. Specific indicators 
of quality are expected to change even 
though the basic conceptual framework for 
elements of quality education will remain the 
same. 

Consequently, the nature of quality 
makes assessment difficult but not impossi- 
ble. The difficulty in no way reduces the 
necessity. Popham and Grasso (cited in 
Grasso 1979) have stated that assessing the 
worth or merit of educational phenomena is 
an essential component of systematic educa- 
tional evaluation. In addition, section 112 
(b)(1)(B) of the 1976 Amendments to the 



Vocational Education Act of 1963 as inter- 
pretrKl by Federal regulations explicitly states 
the need for evaluating **quality and availabil- 
ity of instructional offerings" (Federa/ f?eg/s- 
fer. 3 October 1977; section 104.402). Thus, 
the assessment of quality in vocational edu- 
cation is not only generally recognized and 
accepted as important, but it is mandated by 
law. 



Approaches to Assessing Quality 

Considerable effort has been expended 
in developing evaluative strategies to facili- 
tate the assessment of quality. Much of this 
work has been drawn from the areas of 
organizational behavior and systems theory 
and adapted to specific field evaluation mod- 
els. Three predominant approaches have 
emerged that are relevant to assessments of 
quality within the field of vocational educa- 
tion. We will generally refer to them as struc- 
tural, process, and outcome approaches. 



Structural Approaches 

Structural approaches focus on aspects 
of the organization that impact on quality of 
education. This would include such variables 
as personnel, facilities, equipment, formal 
organizational structure, policies, and so 
forth. These variables primarily relate to 
components and elements of quality within 
the school context domain of our model (fig- 
ures 1 and 2). For example, in a structural 
approach to the issue of human resource 
development, a judgment might be made that 
certain teacher qualifications are necessary 
to ensure quality education. These qualifica- 
tions would be expressed as indicators within 
perhaps the teacher skills and teacher char- 
acteristics components of school context. Or. 
in terms of classroom seff/ng, indicators 
might be developed that specify teacher- 
student ratios and student-square feet ratios, 
and so forth. These would serve as standards 
of quality for instructional setting. 
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Two maior atsump^iont support a struc- 
tural approach to assassing quality. First, it is 
poaalMa to Mantify "good** in tarms of staff, 
physical facilitias. formal organixation. and 
so forth. And sacond, tha quality of aduca- 
tion will improva as a rasult of such things as 
battar qualifiad staff, improvad physical facili- 
tiaa and aquipmant. and soundar organiza- 
tional structura. 



^rocasa Approachas 

Procass approachas utiliza critaria- 
oriantad mathods and individual studias to 
axamina activitias involving taachars. stu- 
dams. matarials and thair intaraction. and 
othar alamar^ts ^hat ara includad within our 
daftnitipn of school procass. Tharafora. pro- 
caas approachas would primarily focus on 
tha school procass portion of our modal (fig- 
ura 3). A procass approach might axamine 
such tt)ir>gs as *'hands»on** aquipment time in 
a walding class, frequency of **teacher ques- 
tioning student** during a lecture class, or 
clarity of course objectives against selected 
indicators of quality. 

Process approaches to the assessment of 
quality assume that it is possible— 

1 . to set intermediate course of program 
goals and evaluate whether goals 
have been met. 

OR 

2. to establish quantitative and qualita- 
tive standards of quality, based on 
expert opinion and applied to indi- 
vidual case review. 

OR 

3. to directly observe critical behaviors 
that permit determination of quality. 

AND 

4. for school educators and administra- 
tors to agree upon what constitutes 
quality without continually monitor- 
ing educational outcome. 

Process evaluations can be utilized to detect 
problems in program design and implementa- 
tion, to provide information for formative 



decision making, and to develop databases 
for long-range planning (Stufflefc^eam et al. 
cited in Grasso 1979). 



Outccma Approadiea 

Outcome approaches to assessing qual- 
ity of education assume that certain direct 
measures of results are related to quality of 
education. Theoretically, within a pure 
outcome framework, the qualifications of 
educators, the structure of the organization, 
facilities and aquipment. and school process 
components and their interactions can be 
disregarded in favor of assessing whether or 
not desired results are achieved. One exam- 
ple of an outcome approach is Scriven*s con- 
ceptualization of payoff evaluation that 
appears to include analyses of both outcome 
and impact (cited in Grasso 1979). 

The underlying assumptions of an out- 
come approach are the following: 

1. There is a high level of agreement 
between societal, individual, and pro- 
fessional estimations as to what end 
results are desirable. 

2. Desired outcomes can be expressed 
as quantitative and qualitative indices 
of success. 

3. Desired outcomes are brought about 
to a great extent by quality 
education. 

4. There is a high level of agreement 
between experts and practitioners 
regarding validity of selected out- 
come indicators 

An outcome approach would focus primarily 
on the elements of quality vocational educa- 
tion outcomes depicted in the right hand 
panel of our model (figure 1). Outcome indi- 
cators or sets of outcome indicators would 
be developed for each component in this 
outcome panel and a determination of 
attainment or lack of attainment would 
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subMqutntly bt mad«. In tome cases, the 
■si ee im ent could be immediate; in others, 
there may be a considerable time lag. For 
example, reeding ability might be assessed 
by means of a reading achievement test bat- 
tery iutt prior to graduation. In some profes- 
sions, licensing exams are taken shortly after 
graduation and serve as indicators of attain- 
ment of iob^specific skills. But in the case of 
positive interpersonal interactions, a consid- 
erat>le period of time on the Job may be 
required prior to assessment and the method 
of measurement may not t>e so readily 
available. 

A major difficulty with a pure outcome 
approach is that it requires no systematic link- 
age between desired outcomes and com- 
ponents and elements of the educational 
experiertce. Program modifications resulting 
from a pure outcome approach may not be 
systematic. The problem is due to the limited 
scope of the analysis. The same problem 
exists for pure structural and process 
approaches. 

An evaluation system designed to assess 
quality of vocational education as defined by 
our conceptual framework must consider 
school context, school process, and out- 
comes and therefore requires a synthesis of 
these approaches. An integrated approach to 
the evaluation of quality assumes that 
appropriate structure (school context) 
increases the probability of quality programs 
(school process), which, in turn, increases 
the likelihood of successful outcome. 

The CIPP (Context. Input. Process. Prod- 
uct) model designed for institutional decision 
making (Stufflebeam et al. cited in Qrasso 
1979) is one example of a integrated evalua- 
tion approach that could be applied to the 
assessment of quality. Another would be 
Drewes* expansion of the CiPP model to 
include impact evaluation (Drawee et al. 
1975). Drawee' comprehensive model incor- 
porates structural, process, and outcome 
approaches within the following evaluation 
methodologies: 



• Context evaluation— identifies unmet 
needs and unused opportunities 

• Input evaluation*— identifies and 
assesses merits of various structural 
variables such as staff, organization, 
and physical facilities 

• Process evaluation*— reviews opera- 
tions of programs and the teaching- 
learning process as a whole 

• Product evaluation*— measures and 
interprets attainments at end of 
courses and programs to assess 
intermediate goal attainment 

• impact evaluation*— examines 
intended and unintended results of 
vocational education, such as 
employee satisfaction and post- 
program placement 

• Palrwiee evaluatlon*-examines pair- 
wise interrelationships of evaluation 
types listed previously 

The primary point is that a comprehensive 
and systematic evaluation of the elements of 
vocational education quality requires a sys- 
tem of evaluation that includes structural, 
process, and outcome approaches. 



Developing Indicators of Quality 

Indicators are standards a lainst which 
something, in this case, quality, :an be mea* 
sured. In order to assess quality ^f vocational 
education comprehensively, sets of indica- 
tors need to be developed for the elements of 
quality presented in chapter 2 (outcomes) 
and chapter 3 (educational experience) and 
incorporated into an evaluation system such 
as Drewes* model. Assessing quality within 
these constraints imposes formidable difficult 
ties in terms of time and cost. It is not practi- 
cal to tabulate exhaustively all of the 
indicators related to the elements of our 
model, nor is it financially nractical to 
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support an tvaluation system capable of col- 
iectino and analyzing such data in a com- 
plete form. 

Our approach to this difficulty involves 
two initial steps: 

1. Prepare a representative list of sub- 
elements. These should be selected 
to encompass the range of each 
major component and element com- 
bination of our model as described in 
chapters 2 and 3. 

2. Secure a consensual judgment about 
the relative priority of time or 
resource investment in the subele- 
ments. These priorities might serve 
as proxies for the overall value of the 
subelements. (Allow the essentiality 
of specific sut)elements to influence 
this priority. Some may be essential 
Bn preconditions or in their own right: 
others may be alternatives.) This 
exercise will also provide the oppor- 
tunity to suggest new subelements or 
to eliminate inappropriate ones. 

These steps should lead to the prioritization 
of elements within our framework. This order- 
ing may vary from school unit to school unit 
and within school units over time. But after 
these two steps are reasonably complete, it 
will be possible to develop a set of indicators 
for at least the high-priority subelements 
within 11.3 outcome and educational expe- 
rience portions of our model. 

A set of underlying assumptions is 
needed to support the development of qua;..y 
indicators. The following assunptions have 
been extrapolated from Zusman and 
Lawson's (cited in Riedel et ai. 1974) work 
related to assessing the quality of mental 
health patient care. 

• indicators of ''quality'* education can 
be characterized and rated. 

• Definition of quality education as 
expressed through indicators 



depends on present knowledge and 

opinion. 

• Definition of quality education as 
expressed through indicators 
changes as scope-of-provider 
responsibility changes. 

• Decisions regarding adequacy of par- 
ticular educational structures and 
appropriateness of particular pro- 
cesses and programs are influenced 
by decision makers^ value systems 
and judgments and are therefore 
ultimately subjective. 

These assumptions take into account the rel- 
ative, judgmental, and changing nature of 
definitions of quality as expressed at the 
indicator level. For the purposes of the ensu- 
ing discussion, these assumptions will be 
considered valid. 



Normative versus Empirical 

Both normative and empirical 
approaches should be drawn upc* for devel- 
oping indicators. Normative indicators would 
be formulated on the basis of an ideal of what 
represents excellence. Methods used to 
develop normative criteria include judgment's 
of experts or highly qualified panels and 
state-of-the-art literature and research. Nor- 
mative standards are based more one body 
of knowledge than actual practice. One prob- 
lem with a purely normative approach is that 
practitioners and educators feel that it is not 
practical from a time. cost, and other per- 
spective to apply idealized standards to 
actual practice. Empirical standards, on the 
other hand, are based on actual practice— 
educational programs and courses. A major 
criticism of purely empirical approaches is 
that standards may appear adequate relative 
to other situations (school units, programs, 
courses) yet fall far short of what is achiev- 
able through application of current knowl- 
edge. A blend of the normative with the 
empirical is the recommended approach to 
developing indicators of quality. Empirical 
observation should provide a baseline that is 
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th«n modified by normative judgment; school 
units and programs should not lose sight of 
the ideal. 

QiMNitHathre versus Qualitative 

There exists a long-standing debate con- 
cerning the advantages of qualitative versus 
quantitative methods of evaluation. However, 
a growing t>ody of evaluation researchers 
holds that the two methods can be utilized 
quite profitably ^gather. Each method has 
its biases, so selection of the method should 
b% based on purpose of the evaluation and 
specific constraints connnected with the 
project. 

There are a numt>er of reasons why a 
combination of qualitative and quantitative 
methods is suited to a comprehensive evalu- 
ation of quality. First, a comprehensive 
evaluation requires analyses of quality of 
structural, process, and outcome elements. 
This variety of analyses typically requires a 
variety of methods. Second, when used 
together for the same purpose, the two- 
method-type indicators can support one 
another and often lead to insights that 
neither one could provide independently. 
And third, utilizing a combination of qualita- 
tive and quantitative methods helps to 
account for the biases that are associated 
with each approach (Cook and Reichart 
1979). Evaluators need to fit the method and 
the indicators to the demands and purposes 
of the particular evaluation; restricting one- 
self to only one of the method types will 
probaoly restrict a comprehensive evaluation 
of quality. 



Eiamples of indicditors 

As previously noted, specific indicators 
of quality are typically not generalizable 
across school units or even school programs, 
so it is not practically possible to tabulate all 
potential indicators of ..ie elements of quality 
in our model exhaustively. Therefore. Just a 
few txamples will t>e mentioned in order tc 
clarify the relationship between indicators 



and model components and elements and to 
demonstrate the level of specificity involved 
at the indicator level. These examples are 
shown in figure 4. 

Of course, these are only a few examples 
of indicators that might be associated with 
components and elements of quality voca- 
tional education. The key point is that indica- 
tors must be measurable or observable and 
considered as good proxies for the context, 
process, or outcome elements that they are 
intended to measure. It is realized that con- 
siderable effort is involved in developing 
indicators of quality. However, if the problem 
is attacked in a systematic and phased 
manner, it is possible for individual school 
units and programs to establish some valu- 
able indicators of quality. 



Summary 

The elements of quality proposed in this 
paper have been derived from a diverse t>ody 
of literature and experience. We have attempt- 
ed to incorporate various findings related to 
attributes of quality education and programs 
into i\ general yet comprehensive conceptual 
framework in order to broaden our under- 
standing of the elements of quality vocational 
education. We hope it will contribute to an 
increased awareness of the major compo- 
nents influencing the quality of education 
This report is intended to inform policy- 
makers of the breadth of the elements of 
quality and to suggest an integrated frame- 
work for assessing quality to policymakers 
and vocational education evaluation 
theoreticians. 

For purposes of this inquiry, vocational 
education was defined as school-related 
activities that occur in secondary schools 
and 1-and 2-year programs in postsecondary 
schools; vocational education was consid- 
ered to be a subset of general, formal educa- 
tion, which in turn is a subset of education 
coming from a variety of life experiences. 
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COMPONENT 



ELEMENTS 



INDICATOR 



TMOIlif Skills 



• Technical Knowledge 

• Technical Experience 



• Teaching Methods 



• Credentials, licenses 

e Teacher must have minimum of 

3 yeen of fulhtime work experi- 
ence in f ieki prior to hire 

• Teacher mutt accumulate X num- 
ber of hours of f ieki txperience 
through co-op orogram or summer 
emptoyment tor eech 5-year teach- 
ing perkxJ 

e Prior to hire, tMcher must have 
20 quarter hours in teaching 
methods and 10 quarter hours 
in history-of-educatk>n course work 
from an accredited college 

e Teacher must take minimum of 

4 quarter hours in tMching nr>eth* 
ods course work from an accredited 
college for each 5 years of teaching 



Competen c y Devttopment e Job-specific Skills 



Program graduate receives a passing 
grade on licensing exam 



Student Utiliiatton of 
Instructional Process 



e Patterns of 
Participation 



A minimum of 65% of vocattonal 
program graduates are classified as 
Cortcentrators arni Limited Concen- 
trators 



Baste Skills 



e Reading 



Program graduate score ranks above 
50% percentile on standardized 
reading comprehensk)n test 



Teecher, Student, and 
Material Interaction 



^ Student, Material 



Over a period of 4 lecture classes, 
student shouki spend no mora 
than 25% of class time reading 
material in class 

In metal-working lab, 2*person 
student teams should be working 
together between 40% and 60% 
of class time 



Figure 4. Examplesof quality indicators 
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Our framework for the elements of qual- 
ity graw out of a multiphaaed process that 
included reviews of related literature, expert 
input, and key informant interviews. The 
literature provided preliminary structure to 
the framework which was further refined and 
defined by expert opinion and key informant 
commentaries. 

Some vocational education has been 
directed by law to be evaluated in terms that 
include job placement in training-related 
employment, employer satisfaction, and qual- 
ity and availability of instructional offerings. 
This project, however, initially explored a va- 
riety of desirable outcomes that may help to 
define high quality vocational education pro- 
grams and subsequently identified the com- 
ponents of the educational environment that 
may support attainment of those outcomes. 
The outcomes were therefore considered as 
goals and the structures and processes 
within the educational environment were 
perceived as means for achieving the goals. 
The resulting framework for the elements of 
vocational education quality (figure 1) was 
conceptualized as a complex environment 
converging on the educational experience, 
driven by desired individual and societal out- 
comes, and influenced by environmental 
inputs such as communitv setting and labor 
market conditions. 



The Framework: Elements of Quality 
Vocational Education 



involves the ability of people to have positive 
and mutually supportive relations with others 
in the home, the workplace, and the com- 
munity. A sense of efficacy is an attitude of 
confidence In one's ability to utilize skills to 
achieve goals. The ability to manage a large 
array of tasks without having to depend upon 
someone else is designated as a sense of 
independence. These are the four general 
elements of quality individual outcomes 
included within our conceptual framework 

Social stability and employer needs are 
the primary elements of quality within the 
societal outcome portion of our framework. 
Social stability does not imply social stagna- 
tion; rather, it means gradual, orderly 
change. Provision of essential services, qual- 
ity of work produces, resources for self- 
support, and societal improvement are the 
subelements that support social stability. 

We also examined a subset of societal 
outcomes specific to employers. To be worth 
the investment from the employer's point of 
view, vocational education should contribute 
to available skills, reduced training time, and 
high productivity. These are the three ele- 
ments of quality related to employers needs. 
Our definition of quality vocational education 
outcomes includes, but is not limited to, 
employment-related skills. Rather, vocational 
education of a positive quality must also 
enhance the development of desirable or 
even essential skills or dispositions that are 
not directly employment related. 



Elements of Quality Outcome 

Two general types of outcomes are 
included in our framework: individual and 
societal with a subset of societal outcomes 
specific to employers. Four elements of qual- 
ity relating to individual outcomes were iden- 
tified arid defined. Competency development 
encompasses the development of work- 
related skills (both occupational and job spe- 
cific). ba:ic skills such as verbal and 
mathematical, and general problem-solving 
ability. Positive interpersonal interaction 



Elementa of Quality within 
the Educational Experience 

Taking a systems perspective, the 
processes and structures that define the ele- 
ments of quality within the vocational educa- 
tion f <pe.:ence should relate to intended 
individual and societal outcomes. Our model 
divides the educational experience into two 
domains, school context and school process 
School context deals with both physical and 
organizational aspects of the school unit. 
Personnel facilities, equipment, policies, and 
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formal organizational structure all fall within 
achooi context. The elements of quality 
within school context that are formally 
inciiMted in our model are school setting, 
clnMroom aeff/ng. feacAer skills, tsschsr 
ch^rscfristics. aruJ peer group chsrsctsris- 
tics. Figuratively spealdng. school context 
provides the setting for school process, 
which provides the "action." Estimations of 
quality of school process would consider 
such things as course content; course* 
specific equipment; interaction between 
teacher, student, and materials; and level of 
formal student involvement in the vocational 
education program. The elements of quality 
formally included within school process are 
fsch0r, studsnt snd mstshsl intersction, 
instructionsl contBnt snd matshsl: and sfu- 
dsnt utilizstion of instructionsl process. 

These elements of quality within the 
educational experience should relate to 
intermediate objectives that, in turn, should 
support attainment of desired outcomes; the 
linkage should be systematic between quality 
of experience and quality of outcome in 
order to maintain and replicate effective pro- 
grams and maice modifications as necessary. 



Assessing the Quality of 
Vocational Education 

The elements of quality within our 
frameworic are generalizable across school 
unit and program. However, the specific indi 
cators of quality associated with them may 
vary across school and program and are 
liicely to vary over time. 



Considerable effort has been expended 
in developing evaluative strategies to facili* 
tate the assessment of quality. Evaluation 
models should combine structural, process, 
and outcome approaches in order to assess 
the range of elements of quality within our 
framewori(. 

Specific indicators of quality need to be 
developed at the individual school unit level. 
It is not practically possible to tabulate 
exhaustively all the indicators related to the 
elements of our model, nor is it financially 
practical to support an evaluation system 
capable of gathering and analyzing such data 
in a complete form. A process for identifying 
and prioritizing subelements within the 
model is suggested as a practical starting 
point for developing indicators. A combina- 
tion of quantitative and qualitative methods is 
suited to a comprehensive evaluation of qual- 
ity, and both normative and empirical 
approaches should be drawn upon for devel- 
oping indicators. 

In summary, this discussion presents a 
justification and set of procedures that will 
permit an integrated consideration of the 
elements of vocational education quality. We 
have advocated considering a broad set of 
elements of quality rather than a narrowly 
defined or segregated set. so that tne evalua- 
tion of vocational education reflects its con* 
tribution to a set of living experiences in 
addition to employment. 
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